Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactoring ledger info with unverified signatures #15023

Merged
merged 4 commits into from
Oct 23, 2024
Merged

Refactoring ledger info with unverified signatures #15023

merged 4 commits into from
Oct 23, 2024

Conversation

vusirikala
Copy link
Contributor

@vusirikala vusirikala commented Oct 20, 2024 •
edited
Loading

Description

LedgerInfoWithUnverifiedSignatures is the data structure used to store verified and unverified signatures of ledger info. This is used for the optimistic signature verification feature. A similar data structure is required for implementing OSV for signed batch info messages. To avoid repeating the code, this PR refactors LedgerInfoWithUnverifiedSignatures into SignatureAggregator.

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@trunk-io Trunk.io
Copy link

trunk-io bot commented Oct 20, 2024 •
edited
Loading

⏱️ 5h 8m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
execution-performance / single-node-performance 2h 29m 🟥 🟥 🟥 🟥 🟩 (+2 more)
execution-performance / test-target-determinator 22m 🟩 🟩 🟩 🟩 🟩 
test-target-determinator 17m 🟩 🟩 🟩 🟩 
check 15m 🟩 🟩 🟩 🟩 
check-dynamic-deps 13m 🟩 🟩 🟩 🟩 🟩 (+1 more)
rust-cargo-deny 11m 🟩 🟩 🟩 🟩 🟩 (+1 more)
rust-move-tests 10m 🟩 
rust-move-tests 10m 🟩 
rust-move-tests 9m 🟩 
rust-move-tests 9m 🟩 
fetch-last-released-docker-image-tag 7m 🟩 🟩 🟩 🟩 
rust-doc-tests 6m 🟩 
rust-doc-tests 5m 🟩 
rust-doc-tests 5m 🟩 
rust-move-tests 5m ⬜ 

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

@vusirikala vusirikala requested review from danielxiangzl and removed request for sasha8 October 20, 2024 03:27
@vusirikala vusirikala added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label Oct 20, 2024
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
zekun000
zekun000 approved these changes Oct 21, 2024
View reviewed changes
consensus/src/pipeline/buffer_item.rs Outdated
@@ -40,18 +40,18 @@ fn generate_commit_ledger_info(
)
}

fn ledger_info_with_unverified_signatures(
fn signature_aggregator(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the name is not very descriptive, create_ or construct_signature_aggregator

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@vusirikala vusirikala enabled auto-merge (squash) October 22, 2024 23:08
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite realistic_env_max_load success on ff11a5b46d2e263c78ae156349b7da4dacd32312

two traffics test: inner traffic : committed: 14111.30 txn/s, latency: 2811.77 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5365440
two traffics test : committed: 99.98 txn/s, latency: 1814.22 ms, (p50: 1500 ms, p70: 1600, p90: 1700 ms, p99: 10500 ms), latency samples: 1720
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.966, avg: 1.550", "ConsensusProposalToOrdered: max: 0.326, avg: 0.297", "ConsensusOrderedToCommit: max: 0.385, avg: 0.371", "ConsensusProposalToCommit: max: 0.677, avg: 0.668"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.09s no progress at version 23058 (avg 0.21s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.37s no progress at version 2706164 (avg 8.37s) [limit 15].
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite compat success on b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> ff11a5b46d2e263c78ae156349b7da4dacd32312

Compatibility test results for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> ff11a5b46d2e263c78ae156349b7da4dacd32312 (PR)
1. Check liveness of validators at old version: b29f09f57e898d8d211c8bc3e303f6e50bba2266
compatibility::simple-validator-upgrade::liveness-check : committed: 17092.08 txn/s, latency: 1975.38 ms, (p50: 1900 ms, p70: 2100, p90: 2400 ms, p99: 3100 ms), latency samples: 554380
2. Upgrading first Validator to new version: ff11a5b46d2e263c78ae156349b7da4dacd32312
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 5586.99 txn/s, latency: 5091.48 ms, (p50: 5700 ms, p70: 6000, p90: 6100 ms, p99: 6200 ms), latency samples: 105400
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 5561.70 txn/s, latency: 5907.46 ms, (p50: 6400 ms, p70: 6600, p90: 7000 ms, p99: 7700 ms), latency samples: 192300
3. Upgrading rest of first batch to new version: ff11a5b46d2e263c78ae156349b7da4dacd32312
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6482.05 txn/s, latency: 4383.06 ms, (p50: 5100 ms, p70: 5300, p90: 5400 ms, p99: 5500 ms), latency samples: 122560
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6571.22 txn/s, latency: 4941.72 ms, (p50: 5300 ms, p70: 5500, p90: 6200 ms, p99: 6900 ms), latency samples: 222980
4. upgrading second batch to new version: ff11a5b46d2e263c78ae156349b7da4dacd32312
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 8616.87 txn/s, latency: 3162.68 ms, (p50: 3300 ms, p70: 3800, p90: 4200 ms, p99: 4500 ms), latency samples: 158480
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 7532.59 txn/s, latency: 4190.04 ms, (p50: 3900 ms, p70: 5100, p90: 6200 ms, p99: 7100 ms), latency samples: 252140
5. check swarm health
Compatibility test for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> ff11a5b46d2e263c78ae156349b7da4dacd32312 passed
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite framework_upgrade success on b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> ff11a5b46d2e263c78ae156349b7da4dacd32312

Compatibility test results for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> ff11a5b46d2e263c78ae156349b7da4dacd32312 (PR)
Upgrade the nodes to version: ff11a5b46d2e263c78ae156349b7da4dacd32312
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1249.56 txn/s, submitted: 1251.36 txn/s, failed submission: 1.80 txn/s, expired: 1.80 txn/s, latency: 2483.41 ms, (p50: 2400 ms, p70: 2600, p90: 3900 ms, p99: 4800 ms), latency samples: 111280
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1301.11 txn/s, submitted: 1303.86 txn/s, failed submission: 2.75 txn/s, expired: 2.75 txn/s, latency: 2396.76 ms, (p50: 2400 ms, p70: 2500, p90: 3600 ms, p99: 5400 ms), latency samples: 113380
5. check swarm health
Compatibility test for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> ff11a5b46d2e263c78ae156349b7da4dacd32312 passed
Upgrade the remaining nodes to version: ff11a5b46d2e263c78ae156349b7da4dacd32312
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1171.92 txn/s, submitted: 1174.75 txn/s, failed submission: 2.83 txn/s, expired: 2.83 txn/s, latency: 2714.43 ms, (p50: 2400 ms, p70: 3000, p90: 4500 ms, p99: 6600 ms), latency samples: 99300
Test Ok

@vusirikala vusirikala merged commit 321bbd2 into main Oct 23, 2024
45 of 48 checks passed
@vusirikala vusirikala deleted the satya/refactor_sig_aggregator branch October 23, 2024 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danielxiangzl danielxiangzl danielxiangzl approved these changes

@zekun000 zekun000 zekun000 approved these changes

@ibalajiarun ibalajiarun Awaiting requested review from ibalajiarun ibalajiarun is a code owner

Assignees
No one assigned
Labels
CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vusirikala @zekun000 @danielxiangzl